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I 



INTRODUCTION 



In 1 


956 , 


tha 


Na val 


Air 


Rework Facility at Pensacola 


started 


a trial 


program 


to 


determine if spectrometric 


analysis 


of 


oil 


samples 


could 


be used to predict aircraft 



engine failures. The Nival Oil Analysis Program (NOAPj 
evolved as a consequence of the success of the trial 
program. The program has been expanded to include 
monitoring virtually all Navy lubricated systems. In 1976, 
NOAP was merged with similar Army and Air Porce programs and 
became the Joint Oil Analysis Program (JOAP) . References 1 
and 2 proAade a more detailed background of the oil analysis 
program. 



The prediction of a 
facilitated by the spectrom 
the lubricating fluid from 
is burned in the spectr 
certain individual wearmsta 
(PPH) are determined by 
emitted. A record maintain 
the results for each wear 
samples. After a sample 
equipment is burned, an 
current and past burns to i 
trend development or 
Depending on which wearmeta 
have developed an abn 
frequently able to pinpoint 
required preventive maint 
custodian. Reference 3 con 
sampling and normal limit 



pending equipment failure is 
etric oil analysis of a sample of 
the equipment. The fluid sample 
ometer and the concentrat ions of 
1 elements in parts per million 
the wavelengths of the light 
ed for each equipment contains 
metal concentre tion from previous 
from a particular piece of 
evaluator reviews the results of 
etermine if there is an abnormal 
abnormal concentration level. 
Is or combination of wearmetals 
ormal trend, the evaluator is 
the source and recommend the 
enance action to the equipment 
tains procedures, intervals for 
s of wearmetal concentrations for 
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some of the equipments monitored by JOAP. 



The normal limits of wearmetai concentrations are not 
precise boundaries. One abnormal sample from a given 
equipment does not necessarily constitute a requirement for 
a maintenance action. Many factors must be considered by 
the evaluator. Typically a shorter sampling interval or an 
immediate resample may confirm an abnormal level or may lead 
the evaluator to conclude that some error has occurred 
(sample comtaminat ion or sample interchange) . Other factors 
are the tolerances within which a laboratory can obtain the 
same results on repeated experiments (repeatability ) , and 
can reproduce either its own or another laboratory's results 
(reproducibility) . These two factors are considered in this 
thesis. 

Intuitively, it can oe seen that a laboratory must be 
capable of obtaining fairly consistent results for repeated 
burns of an oil sample. Otherwise there would be little 
reason to expect the laboratory to provide data from which 
the evaluator would detect a discrepant equipment. He might 
expect also that the procedure would often identify a 
properly functioning sguipment as being discrepant. 
Similarly, because military equipments are frequently 
transferred from one location to another, it is desirable to 
be able to use the results from different laboratories for 
the same equipment without having to wait for each 
laboratory to separately develop trend information about 
that equipment. As a means of insuring that consistent 
results are provided by laboratories, a certification 
program exists in JOAP (reference 3) . The procedures for 
certification consider only an evaluation of a laboratory's 
individual performance. Reference 4 contains recommended 
procedures for laboratory certification involving both an 
evaluation of a laboratory's performance and irs comparison 
with a control laboratory. Table 3-2 of reference 3 
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contains values of maximam allowable accuracy index and 
repeataoility index for laboratory certification. But 
questions have arisen as to whether the values in the table 
refer to repeatability or reproducibility and what is the 
significance level of the values. Because of these 
questions there is some doubt about the validity of the 
numbers. 

This thesis investigates data provided by twenty six 
laboratories that utilize the Baird Atomic AE35/tJ-3 Atomic 
■Smission Spectrometer. The data collected are for 0, 3, 10, 
50 and 100 PPH concentrations for each of twenty elements. 
Each sample analysis is replicated fifteen times and each 
set of fifteen replications is repeated on two different 
days. 
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II 



CONSIDERATIONS IN SPECTROMETRIC OIL ANALYSIS 



A. MEASOEEUENT ERRORS 



As discussed in reference 5, there are numerous 
potential sources of error in results from the atomic 
emissions spectrometer. For each observation taken (for 
each oil sample burned) , the observed reading can be 
considered to include a true PPM reading plus an error term. 
The error term can be modeled as a random variable and, for 
the moment, can be thought of as an accumulation of the 
effects of all the possible sources of error. These sources 
include inputs such as temperature, humidity, electrode gap 
width, spectrometer standardization, inhomogeneous oil 
samples, operator technique, contamination, etc. The 
observed reading can be mathematically represented by: 



Y = u + S 
i i 

where Y is the result of the ith observation, 
i “ 



( 1 ) 



U is the true PPM, and 



E is the error component of the i;^ observation, 
i 



Many of the error inputs can be controlled to some 
degree by careful attention to precedures established for 
the Oil Analysis Program. These include sample handling, 
spectrometer standardization and operator techniques. 
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However, in burning oil samples collected from operating 
equipments, the operator has no prior estimate of the true 
PPM of the sample nor of the error. 

From equation (1), it can be seen that if the magnitude 
of the random error is allowed to be large relative to the 
true PPM, the error could mask a significant change in the 
estimated PPM. Since the error is not measurable, some 
method is needed to measure the performance of a 
spectrometer. 



B. MEASURES OF PERF0RMAN3E 



Since the random error term can be considered to be 
normally distributed (references 4, 5 and 6), good measures 
of spectrometer performance could be expected to involve 
functions of the sample mean and sample variance. To tesr 
whether a given spectrometer is producing results within an 
acceptable error tolerance level, one might make several 
observations on an oil sample of known PPM concentrarion and 
compute the sample mean and sample variance of the 
spectrometer readings. If the computed quantities were 
within some predetermined bounds, the spectrometer could be 
considered to be operating satisfactorily. Otherwise, the 
spectrometer should be realigned electronically 

(restandardized) and retested. The measures of performance 
currently used in JOAP are; 



Al = I p - yl 

and 



(2) 



_ 2 

= (SOM. (y . - y) / (N-1)) 

1 1 



M 



1 1 



RI 



1 



1 



/ • • • / 



(3) 



where y is the known PPl concentration of the oil sample. 



y = SOM (y /N) , i = is the sample mean, 

i i 

y is the ith observation on the sample, 
i 

N is the total number of observations taken, 

AI is the accuracy index (true PPM - sample mean) , and 

RI is the repeatability index (sample standard 

deviation) . 

For the reasons cited above, it was decided that the 
measures of performance represented by equations (2) and (3) 
are satisfactory for use with oil analysis program data. 

Currently used values of the maximum allowable AI and SI 
are given in table 8-2 of reference 3. In their work with 
data obtained form the oil analsis program, D. 3. Barr, T. 
Jayachandran , and H. J. Larson have found that the tabled 
values may not be realistic bounds (reference 4) . 

As current doctrine in the JOAP procedure, an operator 
makes ten sample burns from a standard oil sample of known 
PPM concentration and uses equations (2) and (3) to compute 
AI and RI for each element under analysis. If the computed 
AI and RI do not exceed tabled values, the laboratory is 
considered to be operating within acceptable tolerances. 
Otherwise, the operator should restandardize the 
spectrometer and repeat the sample burns. 
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III. ANA LYT ICAL RE^LTS 



A. OBJECTIVES 



The goal of this project was to statistically esximata 
the bounds or limits for kl and RI for both the within 
laboratory effect (repeatability) for a single laboratory, 
and between laboratory effect (reproducibility) for a given 
laboratory compared with a control laboratory. However, it 
was discovered that three separate and distinct sets of 
indicies could be identified. Appendix B is devoted to a 
development of the three sets of bounds for the indices. 
Notationally , the three situations are identified as Case I, 
Case II and Case III. 

B. RESULTS 



The procedures presented in this section apply to any 
one of the twenty elements for which the AS35/U-3 
spectrometer is used in JOAP. They concern standardization 
with standard oil samples having any one of 0, 3, 10, 50 or 
100 PPM concentrations. The procedures should be applied 
for each element and concentration of interest. 



1 . C ase I 

Case I corresponds to the current usage of AI and 
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r- - 










RI, that is, a laboratory makes a set of S sample burns with 
a standard oil sample and computes All and RI1 using 
equations (2) and (3). The computed quantities (using n = 
10 or n = 15 sample burns) may be compared with the bounds 
contained in table I or IV respectively of the next section. 
If the computed quantities are not greater than the tabled 
values, the laboratory's spectrometer may be considered to 
be operating within acceptable tolerances for Case I. 

All is the magnitude of the deviation of the sample 
mean from the known PPM concentration and might be viewed as 
an indicator of spectrometer alignment or standardization. 
RI1 is a measure of the variability in the spectrometer's 
observations. Excess variability suggests either poor 
operator technique or an erratic spectrometer channel. 



2 . C ase II 

Case II may be considered to be the tolerances 
within which a given laboratory would be expected to 
reproduce its own results with a second complete set of N 
sample burns from the same oil sample. The laboratory could 
complete these procedures using any oil sample. The 
following computations would be made for the two sets of 
sample burns: 



AI2 = If - xl 

and 



(4) 



2 

RI2 = (s /s ) (5) 

y X 



where y is the sample mean computed from the first set of 



sample burns. 
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X is the sample mean computed from the second set of 
sample burns, 

s is RI1 computed from the first set of burns, and 

y 



s is RI1 computed from the second set of burns. 

X 

The computed quantities (using N = 10 or N = 15 

sample burns) from equations (4) and (5) may be compared 
with the bounds given in Table II or V respectively of the 
next section. If the computed value of AI2 from equation 
(4) is not greater than the tabled bound and if the computed 
value of RI2 from equation (5) lias between the reciprocal 
of the tabled value and the tabled value, the laboratory may 
be considered to be operating within acceptable tolerances 
for Case II. 



AI2 is the magnitude of rhe deviation of sample 
means between sets of N observations and might be viewed as 
an indicator of a change in spectrometer standization . RI2 
is a ratio of sample variances and measures a laboratory's 
ability to reproduce its previous results. 

A word of caution concerning the interpretation of 
the index RI2 is in order. During the analyses, it was 
noted that the computed sample standard deviations from data 
sets taken from within a spectrometer appeared to be 
related. The correlation coefficent (appendix C) was 

computed for each PPM concentration and element combination 
and tested for significance (Table VII). Because the 
correlation noted was in general quite high, it should be 
expected that in almost all applications of Case II 

procedures, for RI2, the laboratory will pass. 
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3 . Case III 



Case III may be consideced to be the tolerances 
within which a given laboratory would be expected to 
reproduce the results obtained by a control laboratory. 
Under Case III procedures, both the control laboratory and 
the laboratory under test would complete N sample burns on 
the same oil sample. The following computation would be 
made for the two sets of sample burns. 



AI3 = ly - x| (6) 

and 

z 

RI3 = (s /s ) (7) 

y X 

where y is the sample mean of the control laboratory’s N 
observations , 

X is the sample mean of the test laboratory's S 
observations , 

s is RI1 for the control laboratory, and 

y 



s is RIl for the test laboratory. 

X 

The computed quantities (using N = 10 or S = 15 sample 

burns) may be compared with the bounds given in table III or 
VI respectively of the oext section. If the computed value 
of AI3 from equation (6) is not greater than the tabled 
bound and if the computed value of RI3 from equation (7) 
lies between the reciprocal of the tabled bound and the 
tabled bound, the laboratory under test may be considered to 
be operating within acceptable tolerances for Case III. 
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AI3 is the magnitude of the deviation between a 
control laboratory's and a given laboratory's sample means 
and might be viewed as an indicator of a given laboratory's 
alignment with respect to the control laboratory. BI3 is a 
ratio of sample variances and measures a laboratory's 
ability to reproduce the control laboratory's results. 



C. TABLES 

Tables I through VII are discussed in section III and in 
appendices B and Z. Table VIII is a comparison of the 
number of laboratories for which the data submitted was used 
with the number of laboratories submitting data (see 
discussion) . Table IX is an example of the computer output 
for one of the 100 analysis of variance problems (100 PPM 
and element combinations). Appendices A, B and C have 
developments for most of table IX. Tables I through VII are 
taken from the 100 computer output pages. Table X is also 
taken from the computer output pages and shows for which PPM 
and element combinations the spectrometer effect was 
significant (appendix A) . 
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CASE U All AND RI I FOR N « 10 SAMPLE BURNS 
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10 SAMPLE BURNS 
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TABLE VIII 





MO. OF 


LABS USED 


poM 


0 


3 


FE 


25/2 5 


2 6/26 


AG 


25/2 5 


26/26 


AL 


24/2 5 


23/25 


BE 


21 /2L 


22 /22 


CR 


25/2 5 


26/26 


CU 


25/25 


26 /26 


MG 


25/2 5 


2 6/26 


NA 


22/22 


2 0/23 


NI 


25/25 


26/26 


P8 


24/24 


25/25 


SI 


25/25 


26/26 


SN 


21 /23 


19/24 


TI 


25/25 


26/26 


b 


21/23 


23/24 


BA 


22/23 


24/24 


CO 


22/22 


23/23 


MN 


21/22 


21/21 


MQ 


25/25 


24/26 


V 


22 /22 


20/23 


ZN 


19/21 


19/21 



. OF LABS submitted 



10 


50 


100 


25/26 


26/26 


26/26 


26/26 


26/26 


26/26 


26/26 


26/26 


26/26 


22/22 


21/22 


21/22 


26/26 


25/26 


26/2 6 


26 /26 


26/26 


26/26 


26/26 


26/26 


25/26 


21/23 


21/23 


19/23 


26/26 


26/26 


26/26 


25/25 


24/25 


23/25 


26/26 


26/26 


26/2 6 


21/23 


23/25 


23/24 


26/26 


26/26 


25/26 


23/24 


20/24 


22/24 


23/24 


23/24 


23/24 


22/23 


21/23 


21/23 


22/2 4 


21/23 


21/23 


25/26 


24/26 


24/26 


20/23 


19/22 


21/23 


19/21 


14/19 


14/20 
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TABLE IX 



PPM: 0 ELEMENT: FE 



YIJ, BAR 



LAB 


DAY 1 


DAY 2 


1 


-0.233 


0. C6G 


2 


0.053 


-0 .067 




0.093 


0.007 




-0. 260 


-0.547 


5 


-0.313 


-0,620 


6 


-0. 333 


-0. 047 


7 


-0.667 


0.200 


b 


0.233 


0.300 


9 


0.3A7 


0.227 


11 


-C .300 


-0.320 


12 


-C. 000 


0.153 


13 


C.480 


0.773 


14 


-0.373 


-0.407 


15 


0.120 


-0.120 


16 


-0 .540 


-0.687 


17 


-0.067 


-0.200 


lo 


-0.500 


0.240 


19 


0 .0 


0.0 


20 


-0.073 


-0.053 


21 


0.0 


O.C 


22 


-0.037 


-0.140 


’3 


0. 013 


0. 013 


24 


-0.153 


0. 147 


25 


-0.347 


-0.247 


26 


-C. 427 


0. 060 


GRAND 


MEAN (Y 


... BAR) 



-C.C92 



STANDARD DEVIATION 


DAY 1 


DAY 2 


0.226 


0.2 03 


0.363 


0.232 


0.194 


0 .428 


0.210 


0.270 


0.131 


0.227 


0.238 


0.136 


0.438 


0. 414 


0.206 


0.169 


0.213 


0.258 


0.256 


0.462 


0.314 


0 . 484 


0.457 


0.433 


0.128 


0.139 


0.278 


0.347 


0. 192 


0.239 


0-184 


0.1o5 


0. 125 


0.083 


0.0 


0.0 


0.313 


0.2‘*2 


0.0 


G.O 


0.300 


0.2S2 


0.164 


0.270 


0.136 


0. 458 


0.449 


0.400 


0. 592 


0.447 



CORR COEF 
0.630 



AnCVA TABLE 



SOURCE 


OF 


SUM OF SQUARES 


MEAN 


1 . 


6.3 66 


BETWEEN INSTRUM 


24. 


48.553 


DAY WITHIN INST 


25 . 


16.740 


'.nIthin in strum 


TOO. 


60.971 


total 


750. 


132.630 



MEAN SOIJARES 

2.023 
0.6 70 
0 .03 7 



SOURCE F STATISTIC 

BETWEEN IMSTRUM 3. 02 13 

DAY WiThIN INST 7.68 7 5 

WITHIN INSTRUM 



F VALUE COMP .V AR .ES"^ 
1.9643 0.045 

1.5219 0.039 

0.087 



table values 

CASE I 

AI C.615 

RI C.520 



FOR 10 BURNS 

CASE II CASE III 

0.631 0.869 

4.026 4.026 



TABLE VALUES FOR 15 BURNS 



CASE I CASE II CASE III 



AI C.t>06 0.615 0. 357 

RI C.Ad4 2.979 2.979 
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TABLE X 





SIGNIFI 

#> 


CANT 


SPECTROMETER 


EFFECT 




PPM 


3 


3 


10 


50 


ICG 


FE 


YES 


NO 


YES 


YES 


NO 


AG 


YES 


YES 


YES 


YES 


NO 


AL 


YES 


MG 


YES 


YES 


YES 


BE 


YES 


YES 


YE S 


NO 


NC 


CR 


YES 


YES 


YES 


NO 


NO 


CU 


Y ES 


YES 


YES 


NO 


NO 


MG 


YES 


YE S 


YE S 


YES 


NO 


NA 


NO 


YES 


NO 


Y ES 


YES 


N1 


Y ES 


NO 


NO 


YES 


YES 


PB 


NO 


NO 


NO 


YES 


NO 


SI 


NO 


NO 


NO 


NO 


MO 


SN 


YES 


NO 


YES 


YES 


YES 


TI 


YES 


NO 


NO 


YES 


NC 


6 


NO 


NO 


NO 


NO 


YES 


EA 


Y ES 


YES 


YES 


NO 


YES 


CD 


YES 


NO 


YES 


NO 


NO 


MN 


YES 


YES 


YES 


NO 


NO 


PC 


YES 


NO 


YES 


YES 


NO 


V 


NO 


NO 


YES 


YES 


NG 


ZN 


YES 


NC 


NO 


YES 


NG 
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IV. DISCUSSION AND CONCLUSIONS 



The data 


submitt ed 


was. 


in 


general, assumed to be a 


representative 


sampling 


from 


the 


population 


of AE35/U-3 


spectrometers 


used by 


JOAP. 


Ho 


wever, from 


X vs s plots 



(reference 9) made from the sample means and sample standard 
deviations of the data, it appears that some laboratories 
submitted data which was Qot consistent with the data from 
other laboratories. Assistance, in determining which sets 
of data appeared to be erratic or from spectrometer channels 
with set up (standardization) problems, was obtained from 
the JOAP Technical Support Center. Table 7III shows the 
number of laboratories from which the data was used compared 
with the number of laboratories submitting data. In 
general, only those sets of data for which the laboratory 

appeared to stand apart from the rest on the I vs s plot and 
were identified by the Technical Support Center were removed 
from the analyses. It is felt that a few of the 
laboratories may have been more than conscientious in 
setting up the spectrometer for the sample burns while a few 
others may have been somewhat careless. If this were indeed 
the situation, the variability (caused by the two extremes) 
could have been responsible for the apparent correlation 
noticed (table VII) for many PPM concentrations and element 
combinations. This would contribute erroneously to the sums 
of squares due to error, making the estimates larger in 
magnitude than they should be. 



The tables of estimates are in a crude form: perhaps 

smoothing over elements and PPM concentrations could reduce 
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the coarseness of the values. To use the tables for PPH 
concentrations between those given, the user might use 
either linear interpolation or least sg^uares regressi^on. 
Linear least squares regression was tried using PPM as the 
independent variable ani the estimate as the dependent 
variable. In general, a good fit was obtained. However, 
some of the tabled values appear to be outliers and in those 
cases the results were not usable. Using Bartlett's test 
for equal variances (appendix B) , it was found that the 
assumption of equal (homogeneous) variances was not valid. 
According to Meter and Masserman (reference 9) , unequal 
variances can have pronounced effects on inferences about 
the variance components (appendix A) when using a random 
effects model. 

Future work in this area should include a repeat of this 
project with a qualified observer present at each laboratory 
for the data gathering to ensure that proper standardization 
procedures are followed. In this project, it was found that 
the day (standardization) effect was significant for all PPM 
and element combinations. In many instances, the F 
statistic was more than ten times larger than the tabled F 
value (appendix A) . 

Because the day effect was significant in all cases, it 
is felt that the Case II procedures described in this report 
could be very useful to JOAP. Case I procedures (currently 
used in JOAP) used in conjunction with case II and case III 
procedures might ensure that laboratories could produce 
consistent results on both interlaboratory and 
intralaboratory comparisons. 
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APPENDIX 1 



MAIHEHATICaL MODEL 



An Analysis of yariaace Model (references 7 and 9) is a 
statistical tool utilized by an experimenter to study the 
relationship between a dependent variable (an observation on 
a sample burn) and one or more independent variables (mean 
PPM concentration, spectrometers, standardization setting, 
error, etc) . No assumptions are required about the nature 
of the statistical relation. The effects of the independent 
variables are separately studied by partitioning the total 
sum of squares (the summation of all the observations 
squared) and the associated degrees of freedom into smaller 
sums of squares that are specifically related to the 
independent variables. Dividing a sum of squares for an 
effect by its associated degrees of freedom gives an 
unbiased variance estimate (mean square) . A ratio of two 

variance estimates, under the null hypothesis that the 
particular effect is zero, has an F distribution with 
degrees of freedom v (numerator degrees of freedom) and v 

X 2 

(denominator degrees of freedom) . At a chosen level of 



significance, the test statistic, F (the ratio of variance 

estimates), can be tested for significant effect by 

comparing F with the tabled value of an F distribution with 

V and V degrees of freedom. A value of F greater than or 
1 a 

equal to the tabled value (at the chosen level of 



significance) implies that the effect is significant; 
otherwise, it may be considered to be zero. Analysis of 
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Variance models may be used for for fired effects (levels of 
independent variables under study are the only ones of 
interest) or for random effects (levels of independent 
variables under study are a subset of a population that is 
of interest) . 

The model used in analyses of the data is a Nested 
Random Effects Model (reference 7) . i random effects model 
is considered appropriate because it allows for 
generalization of the results obtained from the randomly 
selected subset of a population (spectrometers) of interest 
to the entire population. Twenty five laboratories were 
chosen from the population of 127. The laboratories were 
selected by drawing uniform random numbers between 1 and 127 
until twenty five laboratories had been selected. However, 
those laboratories based aboard ships and in foreign 
countries were exempted from selection. The time required 
to mail oil samples to the exempted laboratories and for 
them to return the results was thought to be excessive. 

In the random selection of oil laboratories for 
participation in the data collection, the laboratory 
expected to be the control laboratory (JOiP Technical 
Support Center) was not selected. However, it was 
considered desirable to have the control laboratory included 
in the sampling. Hence, data were also collected from 
Technical support Center, making twenty six laboratories in 
total. The addition of the non-randomly selected laboratory 
to the sample should not significantly alter the 
generalization of the results to the entire population of 
spectrometers. rbe nested feature of the model allowed the 
author to investigate the day (standardization) effect 
within spectrometers. 

The model can be expressed mathematically as: 
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/ 



I =U+&-*-B +E 

ijk i ijk 

X / j f k 

where Y is the kth observation from the i th spectrometer 
ijk 

on day j. 



U is the mean effect. 



A is the ith laboratory effect, 
i 



B is the jth day effect within laboratory i, and 

jCiJ 



E is the random error of the kth observation on 

ijk 

instrument i and day j . 



The assumptions of the model are that E is 

ijk 

2 2 
distributed NI( 0 ,cr ), A is distributed MI(0,a ), and B 

1 A j < i > 

2 2 
is distributed NI(0,cr ) for each i. The notation NI(0,o ) 



means normally and independently distributed with mean 0 and 
2 

variance o . 

The assumption of a normal distribution for observations 
on oil sample burns seems reasonable and is documented by 
previous work with oil analysis data (references 4, 5 and 

6) . The assumption of independence is somewhat questionable 
based on previous work. It has been found that there exists 
a dependence between certain elements and a dependence 
between PPM concentrations (references 4, 5 and 6). 
However, in view of the fact that all of the analyses in 
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this report were performed for one PPM concentration and one 
element at a time, the assumption of independence does not 
seem unreasonable. 



The sums of sguares for the model (reference 7) are; 



2 

SSM = IJK(T ) , 



2 2 

SSA = JK*SUM. (Y. - Y ) = SQM. (Y . ) /JK - SSM , 



2 

SS (BU) = K*SUM (Y - Y ) 

i,j ij. i.. 



2 2 

= SOM (Y ) /K - SOM (Y ) /JK , 
i,j ij. i i.. 



2 

SSE = SOM (Y • Y ) 

i,j,lc ijk ij. 



2 2 
= SOM Y - SOM (Y ) /K, 

i, j ,k ijk i, j ij. 



2 

SSTO = SOM Y 

i, j, k ijk 

i"* I,...,!, j ~~ 1,...,J, k*l,...,K (all 

where SSM is the sum of sguares due to the mean effect, 

SSA is the sum of squares due to the laboratory effect. 



SS(BJA) is the sum of sguares due to the day within 
spectrometer effect. 



SSE is the sum of sguares due to the random error 
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effect. 



SSTO is the total sum of squares. 



r 



soa I /IJK, 
i,j,h ijlc 




SDM Y /JK 
j,k ijk 



Y = SOM Y /K , i, j and k are defined in equation (a1) , 
ij. k ijk 

I is the number of laboratories, J is the number of days 
sampled, and K is the number of replications for each 
laboratory and day combination. 

The mean squares for the model (reference 7) are: 

MSA = 3SA/(I-1> , 

MS(B|A) = SS (BIA) , and 

MSB = SSE/IJ(I-1) (a2> 

where MSA is the mean squares due to the spectrometer 
affect, 

aS(B|A) is the msan squares due to the day 

(standardization) within spectrometer effect, and 

MSB is the mean squares due to random error. 

The two test statistics accociated with the model are; 

F = aSA/MS(BlA) Witt (1-1) and I(J-1) degrees of 

A 

freedom and 

F = MS (B| A) /MSB witt I(J-1> and IJ (K-1> degrees of 



B 
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(a3) 



freedom, 

where ? is the test statistic used to test for a 

A 

significant spectrometer effect and 

F is the test statistic used to test for a significant 
B 

day effect. 

The usual estimators for the components of variance are; 



a 



a 



asE, 



2 



a 



B 



(as (B1 A) - aSE) /K, and 



a 



o 



(aSA - as (B| A) ) /JK 



(a4) 



A 
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APPENDIX B 



ESTiailORS FOR AI AND RI 



While developing bounds on AI for the three cases, it 
was found that for variance estimates the usual chi-sguare 
statistic was not appropriate. The sum of it independent 

2 2 

chi-sguare distributed random variates (n 3 /a ) with n 

i i i 

degrees of freedom (reference 8) is a chi-sguare random 
2 2 

variate (nS /a ) with n=n +n + + n degrees of 

12 ]c 



freedom. However, the variance 


estimates 


us e d in 


determining the bounds for 


AI were 


linear 


combinations of 


variances for which Cochran's 


theorem 


is 


not 


appli cable. 


Using the procedures described 


in chapter 17 


of 


reference 8, 


an approximate chi-sguare statistic 


ca n 


be 


formed as 


follows: 










u = ng/G 








(b1) 


2 2 

where n = (SUM g x ) /SUM (g 

i i i i i 


2 

X . /li . ) / 
1 1 


i = 


1 r • 


. . , K, is the 



approximate degrees of freedom. 



G 



2 2 2 
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g = g flSA + g MS(B|A| g MSS, 

i a 3 

X = MSA, X = MS (3| A) , X = MSB and the g 's will be 

12 3 

defined in the development that follows for the variance 
estimates. 

A. CASE I 



A 95 percent confidence bound on All (lU - y 1) , is 

i j- 

1/2 

given by t (.975) g , 
n 

where y is the sample mean of K observations with the ith 

i j- 

spectrometer on day j. 



g is a variance estimate of ( u - T ) 

ij- 

n is the approximate degrees of freedom, and 

t (. 975) is the tabled value of Student's t distribution 
n 

having n degrees of freedom. 



The variance of (y - 7 ) can be found as follows; 

ij • 



V(y-r )=7(T ) = V(SOM y /K) 

ij. ij. k ijJc 



= (1/K ) V(SaM (y + A + B + E ) ) 

Ic i j^ i^ i jk 
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is (from equation (a4) ) ; 



z z z z z 

= 1/K (K a + K CT + SUM V(E ) 
a B k ijk 

2 2 2 

= o + o + a /15, k = 

a 5 

an estimate, g , of V ( ^ - T ) 

i j- 

g= g^MSa + g^MS(Bia) + g^MSE, 
g = 1/JK, g = (J-1)/JK and g = (K-H) /KN, (b2) 

12 3 

where I is the number of spectrometers used in the analysis, 
J is the number of days (2) , 

K is the number of repeated observations, and 

N is the number of sample burns for which tables were 
constructed (10 or 15) . 

The approximate degrees of freedom n associated with the 
variance estimate g is: 



n = [g MSa + g MS(B|a) + g MSE]/ 

12 3 

2 2 2 2 2 2 
[g (MSa) /(I-1) + g (MS(B|A)) /I(J-1) + g (MSB) /IJ(K-I)] 

12 3 



(b3) 



The usual 95 percent ipper confidence bound on H1 1 would 
2 1/2 

be [n(MSE)/ x (-0 5) ] , where n is the degrees of freedom 

n 



associated with the variance estimate, MSE, and x (-05) is 

n 
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the tabled value of a chi-sguare distribution having n 

degrees of freedom. During analysis it was found that this 
upper bound on RI1 was aot realistic when compared with the 
sample standard deviations computed from the data. k 
Bartlett test for equal variances was performed (reference 
9) for several PPM and element combinations. It was found 
that for all combinations tested, the hypothesis of equal 
variances was rejected. Therefore, another method for 
estimating the upper bound on RIl was developed. 

To estimate the upper bound on RI1, the sample standard 
deviations for both days on all spectrometers were used to 
construct an empirical cumulative distribution function. 
The 95 percent upper bound was found by linear interpolation 
on the empirical cumulative distribution function. 

To find the upper bound on RI1 for M = 10 sample burns, 
it is assumed that the upper tail of the distribution of 
sample variances can be approximated by the upper tail of a 
normal distribution. To develop a relation between the 
distributions for 10 aad 15 sample burns, it is further 
assumed that the sample variances have first and second 
moments that are related in the same manner as those of 
chi-square variates. The variance of a chi-square variate 
2 2 

(V S / a ) is 2v (reference 7) , and 
i i i 

2 4 2 2 2 4 

V(S ) = a /V V(v s / a ) = 2 a /V 

1 111 i 

and, from the 95 percent upper bound we have 

2 2 

P(S <RI) = .95 
1 1 

22 4 1/2 22 4 1/2 

or, P((S^- a )/(2 a /vj < (RI^- a )/(2 a /vj ) = .95 
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95 



2 2 ♦ 1/2 
or, $((RI^-cr )/{2 cr ) = . 

2 

where S is the estimate of sample variance for N = 15 

i 

sample burns. 



is the degrees of freedom of the estimate (v = 14) 



and 



$ is the standard normal CDF. 

After equating the above to a similar statement for N = 10 

sample burns, we have: 

1/2 2 1/2 21/2 
RII^^ = ((14/9) RII^ ^- ( 1- ( 14/9) ) a ) 

2 

Replacing cr by its estimate (MSE) gives the upper bound on 
RI1 for N = 10 sample burns. 

B. CASE II 



A 95 percent confidence bound on AI2 (|y - x|) is given 
1/2 

by t (.975) g , where y is the sample mean of the first 
n 

set of H observations, and x is the sample mean of the 
second set of N observations. 



The variance of (T - Y ) can be found as follows: 

ij. ij'. 
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V(Y - Y ) = V(? ) + V(Y ) - 2Cov(Y ,Y ) 

ij. ij'. ij. ij*. ij. ij'. 



Cov(Y ,7 ) = E(I - y ) (Y - y ) = cr , 

ij . ij' . i j . ij ' • A 

a 2 

Hence, 7(Y -7 )= 2 a + 2cr/i5. 

i j . i j ' • B 

ka estimate, g, of 7(7 - 7 ) is (from equation (a4) ) : 

ij. ij'. 



g = g MSA -k g MS(B|A| + g MSE 

1 2 3 



g = 0, g = 2/K, and g = 2 (K-N) /KN (b4) 

12 3 

The approximate degrees of freedom, n, are found using 
equation (b3) with the g ' s as defined in equation (b4). 

L 



2 2 

A 95 percent confidence interval on RI2 (s /s ) is given 

y X 



by; 



1/F (.975) < HI2 < F (.975) (b5) 

ni,n2 ni,n2 

where F (.975) is the tabled value of an F distribution 

n^,n2 

having ni and n^ degrees of freedom. 



Since the number of observations, N, is the same for 
both sets of observations ni = n^ = s-1 = 9 or 14. 
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C. CASE III 



A 95 percent bound on AI3 (|y - i|) is given by 



1/2 

t (.975) g 



where y is the sampla mean of the control laboratory's N 



observations/ and 



X is the sample mean of the given laboratory's N 



observations . 



The variance of (Y - Y ) is found as follows: 

ij. i'j'. 



7{Y - Y ) 

ij. i'j*. 



=■ V(Y ) + 7 (Y ) - 2Cov (Y , Y ) 

ij. i'j'* ij • i' j ' • 



Cov (Y ,Y ) = 0 

ij. i'j'. 



2 2 2 

V(Y -Y )=2a + 2 a + 2cr/15 

i j. i'j'* A B 



An estimate, g, of V (Y - Y ) is (from equation (a4) ) : 

ij. i'j'. 



g = g^aSA + g^aS(BlA) + g^BSS, 
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= 2/JK, = 2(J-1)/JK, 

The appropriate degrees of 

equation (b3) with the g ' s as 

L 

The 95 percent bound on SI3 is 



and = 2 (K-N) /KN (b6) 

freedom, n, are found using 
defined in equation (b6). 

found using equation (b5) . 
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APPENDIX C 



CORRELATION 



During analysis of the data provided by the 
participating laboratories, the sample standard deviations 
were computed and the correlation coefficient (reference 9) 
was computed as follows; 

rnum = SOH x y - SUM x SOM y/N 
i i i i i i 



2 2 2 21/2 
rdenom = ((SOM x - (SOH x ) /N) (SOM y -(SOM y ) /N) ) 

ii ii ii ii 

r = rnum/rdenom ,i=1,..,,I (cl) 

where y is the sample standard deviation computed from the 
i 

data collected on the first day that data was collected for 
each PPM and element combination, 

X is the sample standard deviation computed from the 
i 

data collected on the second day, and 

I is the number of Laboratories submitting data for a 
given PPM concentration. 

Table VII gives the values of the correlation 
coefficient r for each PPM and element combination. To 
determine whether the correlation coefficents computed were 
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significant, the test statistic t was computed as follows: 



1/2 2 1/2 

t = r(I - 2) /(1 - r ) (5) 

Under the hypothesis that the correlation is zero, t is 
distributed as Student’s t distribution with 1-2 degrees 
of freedom (reference 9 ) . Entering a t distribution table 
with the test statistic t and 1-2 degrees of freedom 
yielded the tail areas that are also given in Table VII. If 
one chose .05 as the desired level of significance, any tail 
area listed in Table 7II not greater than -05 would imply 
that the corresponding correlation coefficient is 
significant: there is a significant positive correlation 

between the standard deviations of the first and second 
day's observations at a given laboratory. The discussion 
above assumes that the day one and day two sample standard 
deviations are observations on a bivariate normal population 
(reference 9) . However, sample standard deviations are not 
normally distributed. Hence, the results in table VII 
should only be interpreted as rough indicators. 
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APPENDIX D 



PAailCIPATING LABOHATOBIES 



the following is a list of the laboratories that 
cooperated by making the the sample barns and submitting 
their results for the author's analyses. The numbering 
system has no relation to the oil analyses program. 

Lab no. Location of laboratory 

1 Technical Support Canter, Pensacola 

2 McClellan APB 

3 Hunter AAF 

4 NAS Norfolk 

5 MCAS Cherry Point 

6 Langley AFB 

7 Pease AFB 

8 Laughlin AFB 

9 Bergstrom AFB 

10 NAS Hhidbey Island 

11 Dover AFB 

12 FT Bucker 

13 Dyess AFB 

14 Myrtle Beach iFB 

15 ANG, Jacksonville, FI 

16 Davis Monthan AFB 

17 McChord AFB 

18 Shaw AFB 

19 Eglin AFB 

20 Holloman AFB 

21 Randolph AFB 

22 Minot AFB 
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23 


NAS Alameda 


24 


Ft Hood 




25 


Grandf ork 


AFB 


26 


Ell sworth 


AFB 
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